Predictive Evaluation of Econometric Forecasting Models in Commodity Futures Markets

The predictive accuracy of various econometric models, including random walks, vector autoregressive and vector error-correction models, are investigated using daily futures prices of 4 commodities (the S&P500 index, treasury bonds, gold and crude oil). All models are estimated using a rolling window approach, and evaluated by both in-sample and out-of-sample performance measures. The criteria considered include system criteria, where we evaluate multi-equation forecasting models, and univariate forecast accuracy criteria. The five univariate criteria are root mean square error (RMSE), mean absolute deviation (MAD), mean absolute percentage error (MAPE), confusion matrix (CM), and confusion rate (CR). The five system criteria used include the trace of second moment matrix of the forecast errors matrix (TMSE), the trace of second moment matrix of percentage forecast errors (TMAPE), the generalized forecast error second moment matrix (GFESM), and a trading-rule profit criterion (TPC) based on a maximum-spread trading strategy. An in-sample criterion, the mean Schwarz Information Criteria (MSIC), is also computed. Our results suggest that error-correction models perform better in shorter forecast horizons, when models are compared based on quadratic loss measures and confusion matrices. However, the error-correction models which we consider perform better at all forecast horizons (1 to 5-steps ahead) when models are compared based on a profit maximization loss function. Further, our error-correction model where the error-correction term constructed according to a cost-of-carry equilibrium condition outperforms our alternative error-correction model which uses the price spreads as the error-correction term.